Developing Grounded Goals through Instant Replay Learning

This paper describes and tests a developmental architecture that enables a robot to explore its world, to find and remember interesting states, to associate these states with grounded goal representations, and to generate action sequences so that it can re-visit these states of interest. The model is composed of feed-forward neural networks that learn to make predictions at two levels through a dual mechanism of motor babbling for discovering the interesting goal states and instant replay learning for developing the grounded goal representations. We compare the performance of the model with grounded goal representations versus random goal representations, and find that it is significantly better at re-visiting the goal states when using grounded goal representations

Self-Motivated, Task-Independent Reinforcement Learning for Robots

This paper describes a method for designing robots to learn self-motivated behaviors rather than externally specified be- haviors. Self-motivation is viewed as an emergent property arising from two competing pressures: the need to accu- rately predict the environment while simultaneously wanting to seek out novelty in the environment. The robot’s inter- nal prediction error is used to generate a reinforcement signal that pushes the robot to focus on areas of high error or nov- elty. A set of experiments are performed on a simulated robot to demonstrate the feasibility of this approach. The simulated robot is based directly on an existing platform and uses pixe- lated blob vision as its primary sensor

The Governor Architecture: Avoiding Catastrophic Forgetting in Robot Learning

The governor architecture is a new method for avoiding catatrophic forgetting in neural networks that is particularly useful in online robot learn- ing. The governor architecture uses a categorizer to identify events and excise long sequences of repetitive data that cause catastrophic forgetting in neural networks trained on robot-based tasks. We examine the performance of several variations of the governor architecture on a number of re- lated localization tasks using a simulated robot. The results show that governed networks perform far better than ungoverned networks. Governored networks are able to reliably and robustly prevent catastrophic forgetting in robot learning tasks